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ABSTRACT 

We present a method for isolating a clean sample of red giant branch stars in 
the outer regions of the Andromeda spiral galaxy (M31). Our study is based on 
an ongoing spectroscopic survey using the DEIMOS instrument on the Keck II 
10-m telescope 7 . The survey aims to study the kinematics, global structure, 
substructure, and metallicity of M31's halo. Although most of our spectroscopic 
targets were photometrically screened to reject foreground Milky Way dwarf star 
contaminants, the latter class of objects still constitutes a substantial fraction of 
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the observed spectra in the sparse outer halo. Our likelihood-based method for 
isolating M31 red giants uses five criteria: (1) radial velocity, (2) photometry in 
the intermediate-width DD051 band to measure the strength of the MgH/Mg6 
absorption features, (3) strength of the Nai 8190 A absorption line doublet, 
(4) location within an (I, V — I) color-magnitude diagram, and (5) comparison 
of photometric (CMD-based) versus spectroscopic (Can 8500 A triplet-based) 
metallicity estimates. We also discuss other giant/dwarf separation criteria that 
might be useful in future analyses: the strength of the Ki absorption lines at 
7665 and 7699 A and the TiO bands at 7100, 7600, and 8500 A. Training sets 
consisting of definite M31 red giants and Galactic dwarf stars are used to derive 
empirical probability distribution functions for each diagnostic. These functions 
are used to calculate the likelihood that a given star is a red giant branch star 
in M31 versus a Milky Way dwarf star. By applying this diagnostic method 
to our spectroscopic data set, we isolate 40 M31 red giants beyond a projected 
distance of R = 60 kpc from the galaxy's center, including three red giants out 
at R ~ 165 kpc. The ability to identify individual M31 red giant stars gives us 
an unprecedented level of sensitivity in studying the properties of the galaxy's 
outer halo. 

Subject headings: galaxies: halo — galaxies: individual (M31) — techniques: 
spectroscopic 

1. Introduction 

Hierarchical galaxy formation theories propose that galaxies are built up through the 
accretion of smaller systems (Searle & Zinn 1978; White & Rees 1978). Lately there has 
been significant progress in the investigation of galactic halos, especially in studies focusing 
on the contribution of tidal debris from disrupted satellites. Computational studies, using 
numerical simulations and semi-analytic modelling, have made great strides in understanding 
the properties of halos built up from tidal debris (e.g., Johnston, Hernquist, & Bolte 1996; 
Johnston 1998; Helmi & White 1999; Helmi & de Zeeuw 2000; Bullock, Kravtsov, & Weinberg 
2001; Bullock & Johnston 2005). Recent observational studies have led to the discovery of 
large tidal streams in the Milky Way (MW) halo such as the Magellanic Stream (Mathewson, 
Cleary, & Murray 1974), the Sagittarius stream (Ibata, Gilmore, & Irwin 1994; Majewski 
et al. 2003; Newberg et al. 2003), and the Monoceros stream (Yanny et al. 2003; Rocha-Pinto 
et al. 2003). These observations are adding new insight to the study of halo formation. 

Studies of tidal structures in the halo of the Galaxy present inherent difficulties, as 
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they generally require wide- field surveys given their large angular extent when viewed from 
our vantage point within the disk. The Andromeda spiral galaxy (M31) presents a unique 
opportunity to study a galaxy like our own from an external perspective. In addition, it has 
the advantage of having a high disk inclination angle of 78° (de Vaucouleurs 1958; Walterbos 
& Kennicutt 1988), and at a distance of 783 kpc (Stanek & Garnavich 1998; Holland 1998) 
it is close enough to allow detailed photometric studies of individual stars (e.g., Mould & 
Kristian 1986; Durrell, Harris, & Pritchet 2001, 2004; Bellazzini et al. 2003), even faint main- 
sequence turnoff stars (Brown et al. 2003, 2006), as well as spectroscopy of individual red 
giant branch (RGB) stars (Reitzel & Guhathakurta 2002; Reitzel, Guhathakurta, & Rich 
2004; Ibata et al. 2004, 2005; Guhathakurta et al. 2006; Kalirai et al. 2006b; Chapman et al. 
2006). 

As in the MW, recent studies of M31 have uncovered evidence of past and present 
accretion events. Star-count maps covering a large area of M31 have revealed a 'giant 
southern stream' and many other signs of disturbance in the spheroid and outer disk (Ibata 
et al. 2001; Ferguson et al. 2002). Evidence of tidal disruption in M31's closest satellite 
galaxies M32 and NGC 205 has been presented by Choi et al. (2002). 

In order to investigate the substructure of the outer halo through kinematics and metal- 
licity, we have undertaken a spectroscopic survey of the M31 halo with the Deep Imaging 
Multi-Object Spectrograph (DEIMOS; Faber et al. 2003) on the Keck II Telescope. The 
first results from this survey focus on the giant southern stream discovered by Ibata et al. 
(2001), and are presented by Guhathakurta et al. (2006), Font et al. (2006), and Kalirai 
et al. (2006b). 

The ability to take spectra of individual stars in the halo of M31 opens up many exciting 
avenues for studying its kinematics, structure, and metallicity (Guhathakurta et al. 2006; 
Kalirai et al. 2006b). Radial velocities are the standard method for discriminating between 
RGB stars in M31 and dwarf stars in the MW. However, the radial velocity distributions of 
these two populations overlap. Although modelling the distribution of radial velocities can 
provide a statistical separation of M31 RGB versus Galactic dwarf stars, a more stringent 
method for differentiating the two populations is needed to mark individual stars as M31 
RGB or MW dwarf stars. This is vital for studying the far outer regions of the halo, where 
our survey is expected to find only a handful (if any) RGB stars per DEIMOS pointing, and 
will give us the sensitivity needed to probe M31's halo to extremely low surface brightness 
levels. 

In this paper, we describe a method that uses a combination of spectroscopic and pho- 
tometric information to determine whether a star is an M31 RGB star or a foreground MW 
dwarf star. This allows us to isolate a clean sample of RGB stars in M31. Details of the 
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observations and data reduction are summarized in §2. The selection of training set stars, 
the different spectroscopic and photometric diagnostics, probability distribution functions, 
and the application of a likelihood-based screening technique are described in § 3. In § 4, the 
robustness of the diagnostic method is tested and it is compared to other methods used to 
study RGB stars in M31. The main points of the paper are summarized in § 5. 

2. Data 

2.1. Slitmask Design and Observations 

Objects were selected for Keck/DEIMOS spectroscopy using the M31 photometry/astrometry 
catalogs of Ostheimer (2002). The catalogs were derived from images taken with the Mosaic 
camera on the Kitt Peak National Observatory (KPNO) 8 4-m telescope in the Washington 
System M and T 2 bands and the intermediate-width DD051 band. The DD051 filter is 
centered at a wavelength of 5150 A with a width of about 100 A. It includes the surface- 
gravity sensitive Mgb and MgH stellar absorption features, which are strong in dwarf stars 
but weak in RGB stars. Photometric transformation relations from Majewski et al. (2000) 
were used to derive Johnson- Cousins V and I magnitudes from the M and T 2 magnitudes. 

Due to the sparseness of RGB stars the outer halo of M31, objects were photometrically 
pre-selected in order to maximize the efficiency of the spectroscopic observations. Each 
object was assigned a DD051 parameter, an estimate of the probability of its being an 
RGB star based on its position in the (M — DD051) versus (M — T 2 ) color-color diagram 
(Palma et al. 2003). Objects with point-like morphology (based on the DAOPHOT-based 
morphological parameters chi and sharp) and large values of the DD051 parameter were 
given the highest priority during the design of the DEIMOS slitmask. Background galaxies 
tend to get assigned high values of the DD051 parameter (either because their Mg6/MgH 
features are redshifted out of the DD051 band or because they are dominated by light from 
RGB stars), but all but the most compact ones fail the chi and sharp criteria. Guhathakurta 
et al. (2006) presents details of the spectroscopic target selection and slitmask design for the 
outer halo fields in our survey. 

In addition to outer halo fields, we have obtained Keck/DEIMOS observations of three 
fields relatively close to the center of M31 that have recently been imaged to below the 



8 Kitt Peak National Observatory of the National Optical Astronomy Observatory is operated by the 
Association of Universities for Research in Astronomy, Inc., under cooperative agreement with the National 
Science Foundation 
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main sequence turnoff with the Advanced Camera for Surveys on the Hubble Space Telescope 
(Brown et al. 2003, 2006). These fields are on the southeastern minor axis (Hll), giant south- 
ern stream (H13s), and northeastern major axis (H13d). We do not have DD051/ Washington 
system photometry for these fields so we did not use the P g i an t criterion for spectroscopic 
target selection. The surface density of M31 RGB stars is very high in these inner fields, 
leading to a large RGB to dwarf ratio even without preselection of RGB star candidates. 
Photometry for the Hll and H13s fields comes from MegaCam images taken with the 3.6-m 
Canada-France-Hawaii Telescope (CFHT) 9 in the g' and %' bands. Object detection and 
photometry for these fields was done using SExtractor (Bertin & Arnouts 1996) and the 
instrumental magnitudes were transformed to the Johnson- Cousins V and I bands (Kalirai 
et al. 2006b). Photometry for the H13d fields comes from short Keck/DEIMOS imaging 
exposures in the V and I bands (Reitzel et al. 2006, in preparation). The lists for slitmask 
design in the Hll, H13s, and H13d fields were based on I magnitude and the SExtractor 
morphological criteron stellarity. 

Spectroscopic observations of DEIMOS multislit masks were obtained in fall 2002, 2003, 
2004, and 2005 (Table 1; Fig. 1) using the Keck II telescope and the DEIMOS instrument 
with the 1200 line mm -1 grating. This grating yields a dispersion of 0.33 A pix~ x and the 
spatial scale is 0'/12 pix -1 . For masks observed in fall 2002, the central wavelength setting was 
A8550 A, and the spectra cover the range AA7200-9900 A. This setting had the unfortunate 
effect that one of the lines of the Can triplet occasionally landed in the inter-CCD gap in 
DEIMOS. To avoid this, and to extend the spectral coverage to shorter wavelengths to include 
the A7100 A TiO and A6563 A Ha features, masks observed in fall 2003, 2004, and 2005 had 
the central wavelength set to A7800 A, yielding the spectral coverage AA6450-9150 A. Slits 
had a width of 1", which subtends 4.8 pix. The spectral resolution is slightly better than this: 
3.8 pix = 1.26 A for the typical seeing conditions (Of! 8 FWHM). This resolution corresponds 
to 44 km s _1 at the Can triplet. Our actual radial velocity measurement error is significantly 
smaller than this (see §2.3) — the centroiding accuracy depends on the signal-to-noise (S/N) 
ratio and is typically much smaller than the width of the line (Tonry & Davis 1979). 



9 MegaPrime/MegaCam is a joint project of CFHT and CEA/DAPNIA, at the Canada- France-Hawaii 
Telescope which is operated by the National Research Council of Canada, the Institut National des Science 
de l'Univers of the Centre National de la Recherche Scientifique of France, and the University of Hawaii. 
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Table 1: Details of spectroscopic observations and basic results. 
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a A number of stars were observed on two different masks (§ 2.3). 

b The high number of failed targets in these masks reflects incomplete data reduction for these masks. Our 
success rate for these masks will increase once the remaining slits are reduced. 
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2.2. Spectroscopic Data Reduction 

Spectra were reduced using the spec2d and specld software developed by the DEEP2 
team at the University of California, Berkeley 10 . Spectra were flat fielded and corrected for 
fringing, and a wavelength solution was determined from arc lamp exposures. Each slitlet 
was sky subtracted, and individual exposures were averaged using cosmic-ray rejection and 
inverse-variance weighting. Extracted one-dimensional spectra were cross-correlated against 
stellar and galaxy templates spanning a range of spectral types to determine the redshift. 
The ten best cross-correlation matches were reported by the specld pipeline. These matches 
were evaluated by eye using the visual inspection software zspec [developed by D. Madgwick 
and one of us (M.C.C.) for the DEEP2 survey] and the best one identified. In rare cases, the 
correct redshift did not appear among the top ten cross-correlation matches in spite of the 
presence of one or more visually identifiable spectral features [such as the Can triplet, Nai 
doublet (dwarf stars), and A7100 A TiO band]; the redshift had to be determined manually 
in these cases. Each spectrum was assigned a quality code: spectra with secure redshifts 
was assigned Q — 4 (at least two good spectral lines) or Q = 3 (one good line and one 
marginal line). Objects for which the spectra did not yield a reliable redshift were assigned 
Q = 2 (visible continuum with low S/N or no lines), Q — 1 (very weak and/or undetectable 
continuum), or Q = —2 (catastrophic instrument failure). We make no distinction between 
Q = 4 and Q = 3 spectra in our analysis. Table 1 lists the number of targets per quality bin 
for each mask. For more details about the reduction of the survey data see Guhathakurta 
et al. (2006). 

Spectra which have secure redshift measurements (Q = 4 or 3) are shifted to the rest 
frame and rebinned onto a common, uniform (linear) wavelength grid. The continuum 
level of each spectrum is normalized to unity and the S/N computed in the region of the 
Can triplet (both the overall S/N and the S/N between strong atmospheric emission lines). 
The S/N calculation is based on the assumption of Poisson errors as reported by spec2d 
for each wavelength element; the lack of flexure and fringing in DEIMOS allows one to 
achieve Poisson-limited sky subtraction. Finally, the spectra are smoothed by a 10 pixel 
boxcar in wavelength weighted by their inverse variance. Line strength [equivalent width 
(EW)] measurements are performed on the normalized spectra using a measurement window 
centered on the absorption feature (e.g., Nai doublet at 8190 A) with continuum windows 
on each side of the feature (see Table 3). Details of the EW measurements are provided in 
the individual diagnostic sections (e.g., §3.2.4). The measured stellar radial velocities are 



10 http : / / astron . berkeley . edu/ ^cooper/ deep/ specld/primer . html, 
http : / /astron . berkeley . edu/^cooper/deep/ spec2d/pr imer . html 
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corrected to heliocentric velocities using the IRAF 11 task rvcor. 



2.3. Empirical Error Estimates from Duplicate Measurements 

In some of our survey fields, masks were designed to overlap, allowing for repeat ob- 
servations of a total of 84 objects. The duplicate measurements allow us to make empirical 
estimates of the errors in the velocity and line widths measured from the spectra. Most of 
the 84 duplicate measurements with successful redshifts are from field H13d (61). There are 
9 duplicate measurements with successful redshifts in field aO, 8 in field a3, and 2 each in 
fields Hll and H13s. For each pair of spectra, the difference in the EW estimates (or radial 
velocity estimates) between the two masks is calculated. The measurement error is assumed 
to be proportional to the inverse of the S/N of the spectrum: 

«= C g) ( . (D 

We define 

so that for the correct value of C, the distribution of X^-'s will be a Gaussian of unit width. 
For each quantity measured from the spectra, Table 2 lists the error in that quantity for a 
S/N of 10, which is typical of the spectra in our sample (see Table 3 for index definitions). 

In addition to providing empirical error estimates for line width and radial velocity 
measurements, the duplicate spectra were co-added to achieve higher S/N for measurement 
of the spectral features. During coaddition, the individual spectra were weighted according 
to the measured inverse variance for each pixel. 



3. Method for Selecting a Clean Sample of M31 Red Giants 

To isolate a clean sample of RGB stars, we calculate empirical probability distribution 
functions (PDFs) based on the properties of known RGB and dwarf stars. Each star is 
assigned individual probabilities of being an M31 RGB or a MW dwarf star based on its 



n IRAF is distributed by the National Optical Astronomy Observatory, which is operated by the Associa- 
tion of Universities for Research in Astronomy, Inc., under cooperative agreement with the National Science 
Foundation. 
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Table 2: Empirical error estimates of quantities measured from the spectra, C (N/S), for our 
typical S/N of 10. 



Quantity 


Typical Error 


Velocity 


17 km s _1 


Nai EW 


0.74 A 


Can EW (ECa) 


1.09 A 


[Fe/H] spec 


0.46 dex 


K 7665 EW 


1.4 A 


K 7699 EW 


0.6 A 


TiO 7100 EW 


0.07 A 


TiO 7600 (l) EW 


0.05 A 


TiO 7600 (2) EW 


o.i A 


TiO 7600 (3) EW 


0.3 A 


TiO 7600 (comb) EW 


0.2 A 


TiO 8500 EW 


o.i A 



location within each of the diagnostics. The probabilities from the first five diagnostics 
are then combined to give an overall likelihood that the star is an RGB star in M31 or a 
foreground dwarf star contaminant. The selection of RGB and dwarf star training sets, the 
diagnostics, construction of the PDFs, and the final likelihood-based method are described 
below. 



3.1. Training Set Selection 

The DD051-based preselection procedure we have used to select spectroscopic targets is 
very effective at eliminating foreground MW dwarf stars in the color range 1 < (V — I)q < 2, 
but is less effective outside this color range (see §4.1.1 and §4.1.2). This non-uniformity in 
the dwarf rejection probability as a function of color would be difficult to characterize in a 
model training set. We have therefore chosen to use empirical RGB and dwarf training sets. 
Since our PDFs for each diagnostic are empirically determined, the selection of unbiased 
and largely uncontaminated training sets is important. Initial training set stars were drawn 
from all fields observed during the 2002-2004 observing runs except field HI 3d, due to the 
difficulty in separating out M31 disk stars from Galactic dwarfs on the northeastern major 
axis (disk field). 

The initial RGB training set stars were picked mainly on the basis of radial velocity: v < 
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—300 km s" 1 (the systemic velocity of M31). While it is possible for MW halo stars to have a 
heliocentric radial velocity this negative, the expected fraction in our overall data set is very 
small. The inital dwarf training set had to be chosen more carefully, since the radial velocity 
distribution of M31 spheroid stars significantly overlaps the dwarf velocity distribution. We 
included all stars which had been commented as having strong Nai absorption during the 
data reduction phase. Stars with radial velocities v > —200 km s^ 1 in fields with a clear 
RGB/dwarf bimodality in the radial velocity distribution were also included in the dwarf 
training sets. Finally, we included all the stars in field mil, since its large distance from 
M31 (R ~ 165 kpc in projection) makes it extremely unlikely for it to contain RGB stars 
based on previous estimates of the surface brightness profile of M31's halo (Pritchet & van 
den Bergh 1994). 

The initial training sets were used to calculate preliminary PDFs and probabilities for 
all stars from the 2002-2004 observing runs. We used these preliminary RGB and dwarf 
probability estimates to isolate stars that have a much higher combined probability (summed 
over the first five diagnostics) of being an RGB star than a dwarf star. These stars comprise 
our final RGB star training set. A similar procedure was used to construct our final dwarf 
star training set. These final RGB and dwarf star training sets, consisting of a few hundred 
stars each, were used to refine the PDFs for each diagnostic. 

Given that a simple velocity cut alone is not enough to isolate a clean sample of M31 
RGB stars in the outer halo, it is worth explaining why we use a velocity cut to select stars 
for the (initial) RGB training set: 

• The velocity cut used to define the initial RGB training set, v < —300 km s^ 1 (v < 
■u^g 1 ), is very extreme. In fact, only half of all the M31 RGB stars in our sample 
are expected to survive this cut. This is acceptable for the RGB training set, which 
only needs to be a representative set of M31 RGB stars, not necessarily a complete 
one. However, such a cut would adversely affect our ability to find (and analyze the 
dynamics of) RGB stars in M31's sparse outer halo. 

• There is a small but finite probability (< 1%) of encountering a MW dwarf whose 
radial velocity is v < —300 km s _1 . In the outermost fields (e.g., field mil), where 
our sample includes of order 100 MW dwarfs, this can translate to a few dwarf stars 
satisfying the velocity cut. This is of the same order as the number of M31 RGB stars 
found in these fields (see §3.5, §4.4, and Table 5). Thus, even this extreme velocity 
cut would lead to a large MW dwarf contamination rate if we were to use it to search 
for M31 RGB stars in the outer fields. By contrast these few MW dwarfs comprise a 
negligible fraction (« 1%) of the RGB training set; most RGB training set stars come 
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from the inner fields where the surface density of M31 RGB stars greatly exceeds that 
of MW dwarf stars. 

To summarize, the initial training sets were based on extreme cuts in a single parameter: 
radial velocity, Na line strength, or sky position. This resulted in limited, slightly contam- 
inated, and somewhat biased sets of RGB and dwarf stars. By contrast, the final training 
sets were based on cuts in the combined probability ratio. Since this probability ratio com- 
bines information from various parameters, the final training sets are more complete, less 
contaminated, and less biased than the initial training sets. 

The plots for the first five diagnostics below (Figs. 2-5) show the initial training set 
stars overlaid on the final PDFs. The fact that our PDFs did not change drastically when 
we switched from the initial to final training sets suggests that our training sets are stable. We 
investigate our MW dwarf star training set for possible biases by exploring the relationships 
between the different parameters in §4.2.3. The different properties of M31 RGB training 
set stars — Nai EW, Ki EW, TiO EW, (V - I) color, DD051 parameter, CMD position, 
and [Fe/H] p hot versus [Fe/H] spec — have no detectable dependence on radial velocity. 

3.2. Diagnostics 

3.2.1. Method for Constructing Analytic Probability Distribution Functions 

For each diagnostic, analytic PDFs are calculated for each of the M31 RGB and MW 
dwarf star training sets. In this section we describe the general process of computing analytic 
PDFs; subsequent sections describe the specific functions used for each diagnostic. For the 
one-dimensional distributions (radial velocity and DD051 parameter) an analytic function 
was fit to the training set's distribution and normalized. The PDFs for the two-dimensional 
distributions (Nai EW, K i EW, and TiO EW vs. (V-I) color, CMD position, and [Fe/H] phot 
versus [Fe/H] spcc ) are also determined using an analytic fitting scheme. The distribution of 
the data along both the x and y axes must be included in the final PDF. In modeling the two- 
dimensional PDF, the y distribution is treated as a Gaussian whose mean and rms depend 
on x. For each training set, we compute the mean and rms of the y distribution for several 
bins in x: (y)i and of as a function of Xi, where Xi is the mean value in each bin. Analytic 
functions are then fit to the measured (y)i and of values, which are denoted as (y)fit(x) 
and crg t (x), respectively. This describes how the training set's y values are distributed as 
a function of x, but does not contain information about the distribution of x values of the 
training set. We therefore also fit a smooth analytic function P(x) to the projection of 
the training set distribution along the x axis. Finally, the analytic two-dimensional PDF is 
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defined as: 

PDF(a;, y) = C P{x) exp [ - 0.5 [y - (y) Rt (x)] 2 / [a^x)} 2 
where C is a normalization constant such that the PDF integrates to unity. 



(3) 



3.2.2. Radial Velocities 



The radial velocity PDFs and histograms of the training set data are shown in Figure 2. 
Due to the DD051 pre-selection technique employed in our survey (§ 2.1), dwarf distributions 
from standard Galactic models (e.g., Bahcall & Soneira 1984; Ratnatunga & Bahcall 1985; 
Reitzel & Guhathakurta 2002) cannot directly be used for our data set. The dwarf training 
set shows an asymmetric distribution, and the sum of two Gaussians was fit to the heliocentric 
radial velocity distribution of the dwarf training set (§2.2). The initial RGB training set 
was chosen on the basis of radial velocity measurements, therefore it could not be used 
to define the RGB velocity PDF. In addition, many of the observed fields have significant 
substructure or small number statistics, making it difficult to choose an alternate training 
set without biases. Instead, we adopted a Gaussian centered on the systemic velocity of M31 
to define the RGB radial velocity PDF: (ty> = -300 km s" 1 and a = 85 km s' 1 . This 
is based on a fit to our (admittedly limited) sample in Hll, our smoothest spheroid field. 
As shown in Figure 2, there is significant overlap between the RGB and dwarf distributions, 
which makes it difficult to differentiate between RGB and dwarf stars on the basis of radial 
velocity alone. We will return to this issue in § 4.4. 

Our adopted velocity dispersion of 85 km s _1 is probably not representative of the 
entire M31 spheroid; previously published values are at least as large as this (Reitzel & 
Guhathakurta 2002; Guhathakurta et al. 2006; Chapman et al. 2006). The velocity dispersion 
of M31's outer halo (R > 50 kpc) remains poorly constrained at the present time. It should be 
noted that the Gaussian sigma of 85 km s _1 is only used for the definition of the RGB radial 
velocity PDF; we are not suggesting that this be used as a model for the M31 bulge/halo. Use 
of a larger sigma value would result in a higher value for the RGB PDF in the overlap region 
between MW dwarf and M31 RGB velocity distributions. Thus, our adopted dispersion of 
85 km s _1 represents a conservative choice. 

In fields where a skewed distribution of M31 RGB velocities results in greater overlap 
with the MW dwarf velocity distribution, the RGB velocity PDF can be modified to account 
for this. For example, the stellar disk of M31 is apparent as a dynamically cold peak in the 
radial velocity histogram of our NE major-axis field H13d. We use the sum of two Gaussians 
to model the M31 disk and spheroid populations in that field. The rest of our fields show 
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no evidence of a contribution from the extended stellar disk discussed by Ibata et al. (2005). 
This is not surprising given that most of our fields are located at large distances from M31's 
center and/or lie close to its SE minor axis (Kalirai et al. 2006a). 

3.2.3. DD051 Distribution 

The location of a star in (M — DD051) versus (M — T 2 ) color-color space was used to 
assign it a DD051 parameter (§2.1; Palma et al. 2003). Values of the DD051 parameter close 
to 1 indicate that the star is probably an M31 RGB star while values close to indicate it is 
probably a foreground MW dwarf star. The DD051 parameter was used to assign priorities 
to objects during the slitmask design, and it can also be used to help differentiate between 
RGB and dwarf stars independent of the spectroscopic data. (In order to fill the masks to the 
fullest extent possible, some objects with low DD051 parameters were observed.) Figure 3 
shows the PDFs and the initial training sets. The dwarf training set was fit by a double 
exponential, and the RGB training set was fit by a single exponential. For both dwarf and 
RGB stars, the training sets are reduced in number relative to the other diagnostics because 
DD051 photometry was not available for fields Hll, H13s or H13d. As in the case of the 
radial velocity diagnostic, there is some overlap between the RGB and dwarf PDFs. 

3.2.4. Na I Equivalent Width 

The EW of the Nai absorption line at A = 8190.5 A is dependent on both surface gravity 
and temperature, making it likely to be a useful dwarf/RGB star discriminator (Schiavon 
et al. 1997). Table 3 contains details of the EW measurement. 

This diagnostic relies on the distribution of stars in (V — 1) color versus Nai EW space. 
Distributions in each quantity were fit analytically. The dwarf star training set's (V — I) 
color distribution was fit by a double Gaussian and the RGB star training set's (V — I) color 
distribution was fit by a Gaussian plus an exponential. Each training set was then divided 
into bins according to (V — I)q color and the mean and rms of the Nai EW was calculated 
for each bin. Piecewise linear fits were made to the mean Nai EW vs. (V — I)q and to the 
rms Nai EW vs. (V — I)o- The Nai EW distribution can be characterized by Gaussians with 
running mean and sigma values based on the linear fits. The analytic fit to the (V — I)q 
distribution was multiplied by the (running) Gaussian fit to the Nai EW distribution to 
construct the two-dimensional PDF. In other words, x and y were set to (V — I) color and 
Nai EW, respectively, in the context of §3.2.1 and Eqn. 3. 
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Table 3: Absorption line-strength index definitons used in this work. 



Index 


Type a 


Main 
Bandpass (A) 


Continuum 
Bandpasses (A) 




Nai 


g a 


8179-8200 


8130-8175, 8210-8220 




Ca II8498 


g a 


8489-8507 


8560-8644 




Ca II8542 


g a 


8533-8551 


8560-8644 




Cail 8 662 


g a 


8653-8671 


8560-8644 




K7665 


g a 


7660-7670 


7677-7691, 7728-7741, 7764- 


-7784, 








7802-7825, 7877-7895 




K7699 


g a 


7694-7703 


7677-7691, 7728-7741, 7764- 


-7784, 








7802-7825, 7877-7895 




TiCvioo 


c a 


7055-7245 


7012-7048, 7512-7576 




Ti0 760 o(l) 


c a 


7700-8000 


7500-7580, 8080-8170 




Ti0 76 oo(2) 


c a 


7736-7810 


7500-7580, 8080-8170 




TiO 7600 ( 3) 


g s 


none 


7721-7748, 7802-7826, 7878- 


-7895, 








7963-7986, 8001-8024, 8086- 


-8113 


TiOssoo 


9 s 


none 


8474-8484, 8563-8577, 8619- 


-8642, 








8700-8725, 8776-8792 





a Types c a, g a and g s refer to classical-atomic, generic-atomic, and generic-slope-like indices, respectively. 

Figure 4 shows the iso-probability contours for the dwarf and RGB PDFs, along with the 
position of the training set stars overlaid on the contours. Although there is some overlap in 
this diagnostic as well, there is clear differentiation between dwarf and RGB stars at (V — J) 
colors redder than ~ 2, with dwarf stars showing increasingly strong Nai absorption. 

3.2.5. CMD Position 

Position in the (/, V — I) CMD provides yet another dwarf /RGB discriminator. The 
locii of RGB stars, spanning a wide range of metallicities and ages but all at the same 
distance from us, form a well-defined shape in the CMD as delineated by fiducial model 
RGB tracks (Fig. 11). By contrast, dwarf stars form a broad swath in color-magnitude space 
as they tend to be spread out over a wide range of line-of-sight distances. They also tend 
to have brighter / magnitudes. To pursue this as a diagnostic, we defined two parameters 
to describe a star's position in the CMD: Ycmd is the distance of a star along an isochrone, 
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running from at Jo = 22.5 (limiting magnitude of our spectroscopic sample) to 1 at the tip 
of the RGB, and X CM d is the distance of a star on a line drawn across the set of isochrones, 
running from at the most metal-poor isochrone used in our analysis, [Fe/H] = —2.3, to 1 at 
the most metal-rich isochrone [Fe/H] = +0.5. The lines across the isochrones are drawn by 
connecting points of roughly equal mass bins. Values of Fcmd < denote stars fainter than 
Jo ~ 22.5, and values of Fcmd > 1 denote stars above the tip of the RGB (values for these 
stars are calculated by linear extrapolation of the isochrones). Likewise, values of X CM d < 
denote objects bluer than the most metal-poor isochrone, and values of Xcmd > 1 denote 
objects redder than the most metal-rich isochrone (calculated by non-linear extrapolation 
of the lines drawn through the isochrones). The grid of isochrones used in this analysis are 
from Vandenberg et al. (2006) and correspond to an age of t — 12.6 Gyr and [a/Fe] = 0. 

The distributions in X CM d and Fcmd are combined to make the final two-dimensional 
PDF. In the context of §3.2.1 and Eqn. 3, setting x to F C md and y to X C md (instead of 
vice versa) lends itself more naturally to our PDF construction scheme. Specifically, single 
Gaussians were fit to the Fcmd distributions for the RGB and dwarf training sets. The stars 
were then divided into bins according to their FcMDvalues, and running Gaussian parameters 
were fit to the Xcmd distributions. For each of the RGB and dwarf star training sets, the fits 
to the Fcmd and X CM d distributions were multiplied to make the final two-dimensional PDF. 
Figure 5 shows the iso-probability contours for the PDFs and the dwarf and RGB training 
sets. There is a lot of overlap in this diagnostic, but there is still some differentiation between 
the two populations. Since the RGB stars follow the isochrones, they form a rectangular 
distribution in X CM d and F CM d space. In contrast, the dwarfs are evenly distributed in the 
(I, V — I) CMD, and since the metal-rich isochrones curve to redder colors, the Xcmd values 
of bright dwarf stars become progressively smaller with increasing Fcmd values. 

We have chosen to frame the CMD diagnostic in terms of (X CM D! F CM d) instead of the 
purely observational parameters (I, V — I). While the values of X CM d and F CM d are model 
dependent, the separation between the RGB and dwarf PDFs is independent of the details 
of the theoretical RGB tracks used to compute Xcmd and Fcmd- For example, changing 
the adopted age or [a/Fe] for the model isochrones would introduce a small change in the 
Xcmd (and [Fe/H] phot ) values but the change would be the same for RGB and dwarf stars at 
any given location in the CMD. Neither the separation between the RGB and dwarf PDFs 
nor their widths would be affected by such a change. The effectiveness of any diagnostic in 
isolating M31 RGB stars from MW dwarf stars depends on the separation between the two 
PDFs relative to their widths. The distribution of M31 RGB stars in (X CM d, F CM d) space is 
more physically meaningful than their CMD distribution: for example, there is a one-to-one 
mapping from Xcmd to [Fe/H] p hot- Moreover, Xcmd and Fcmd are independent of each other 
in the RGB PDF and this makes it particularly easy to construct the two-dimensional PDF 
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within this space. 

3.2.6. Photometric versus Spectroscopic Metallicity Estimates 

The photometric metallicities ([Fe/H] estimates) for our data set are based on the com- 
parison of a star's position in the (J, V — I) CMD with the above grid (AZ = 0.0001) of 
RGB isochrones at the distance of M31 (Vandenberg et al. 2006). Since the photometric 
metallicity estimates are based on the assumption that the stars are red giants at the dis- 
tance of M31, it is inherently incorrect for dwarf stars. Hence, we expect a clear separation 
between the photometric and spectroscopic [Fe/H] estimates for giants and dwarfs. 

Spectroscopic estimates for [Fe/H] are based on measurement of the Can triplet at 
A ~ 8500 A. The EW of the Can triplet (ECa) was calculated using a linear combination 
of the EWs of the three lines that has been shown to maximize the S/N of the feature 
(Rutledge, Hesser, & Stetson 1997a): 

ECa = 0.5 EW(A8498 A) + 1.0 EW(A8542 A) + 0.6 EW(A8662 A). (4) 

The EW of the individual Can lines was measured in 18 A wide bins. [Fe/H] spec was calcu- 
lated from ECa using an empirical calibration relation: (Rutledge et al. 1997b): 

[Fe/H] spcc = -2.66 + 0.42 [ECa - 0.64(V HB - V)]. (5) 

The last term in the equation, (Vhb — V), corrects for the effect of surface gravity based 
on the difference between the V band apparent magnitude and the apparent magnitude of 
M31's horizontal branch (Vhb = 25.17) (Holland et al. 1996). This calibration relation is 
derived from RGB stars in MW globular clusters spanning a range of metallicities. 

In the construction of the PDFs, x and y were set to [Fe/H] p h t and [Fe/H] spec , respec- 
tively (§3.2.1). Analytical fits were made to the dwarf and RGB training set [Fe/H] p h t 
distributions. Each training set was then divided into [Fe/H] p hot bins, and Gaussians were 
fit to the [Fe/H] spec distribution in each bin. A quadratic function was fit to the Gaussian 
parameters of each of the RGB and dwarf star training sets. The [Fe/H] spcc distribution was 
constructed based on the running Gaussian parameters given by the quadratic fit. For each 
training set, the analytic fits to the [Fe/H] phot and [Fe/H] spcc distributions were multiplied 
to make the final two-dimensional PDF. Figure 5 shows the PDF contours and training set 
distributions in [Fe/H] spcc vs. [Fe/H] pho t- As expected, the M31 RGB stars lie close to the 
one-to-one relation while the dwarf stars are well removed from it; there is a clear difference 
between the RGB and dwarf populations. 
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In using the above calibration relation, we are implicitly assuming that RGB stars in 
M31 are similar to those in MW globular clusters in terms of having old ages and [alpha/Fe] ~ 
+0.3. The latter is different from our value of [a/Fe] = adopted in deriving the [Fe/H] p h t 
estimates for this diagnostic. In their study of M31's extended bulge, Brown et al. (2003) 
find that in addition to the majority old population there is a significant intermediate-age 
(6-8 Gyr) population present. Any difference between our adopted parameters and the 
unknown age and a enhancement of M31's halo RGB stars will cause our [Fe/H] phot and 
[Fe/H] spec estimates to be offset from their true values. However, as discussed in § 3.2.5, such 
offsets will not affect the effectiveness of the diagnostic. For a more detailed treatment of 
the M31 RGB metallicity estimates, we refer the reader to Kalirai et al. (2006a). 



3.2.7. Other Diagnostics 

In this section we discuss two diagnostics based on K i and three diagnostics based on 
TiO. These five diagnostics have a lot in common with the Nai diagnostic and with one 
another (§4.2.1). The three TiO diagnostics have a smaller separation between RGB and 
dwarf stars than the Nai or Ki diagnostics. Our present method is set up for combining 
diagnostics that are independent of one another so we have decided to exclude the K i and 
TiO diagnostics from the overall likelihood scheme (§3.3). We include them in this section 
because planned improvements to our diagnostic method will enable us to incorporate them 
in the future (§4.3.1). 

K I Equivalent Width 

The EWs of the Ki absorption lines at 7665 and 7699 A, like the EW of Nai, are 
dependent on temperature as well as surface gravity, making them potential dwarf/giant 
discriminators. The continuum bandpasses and main bandpasses used to measure the EWs 
of the K i absorption lines are listed in Table 3. The construction of the two-dimensional 
RGB and dwarf PDFs for each of the two K i lines followed exactly the same procedure as 
for the Nai line (§3.2.4). 

The iso-probability contours and the RGB and dwarf training sets are shown in Figure 7. 
As in the Nai diagnostic, the M31 RGB and MW dwarf distributions overlap at bluer colors, 
but diverge at redder colors ((V — I)q > 2.5), with the strength of the Ki absorption lines 
increasing with increasing (V — I) for dwarf stars. 



Strength of TiO Bands 
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Absorption molecular bands are the strongest spectral features of cool stars. In particu- 
lar, TiO bands are apparent in early M-type stars and their strength increases with decreasing 
temperature. For a given temperature, TiO bands of RGB stars are stronger than those of 
dwarf stars, making these spectral features useful dwarf/giant discriminators in the low tem- 
perature regime. A detailed explanation of the behaviour of TiO bands (e.g., those around 
A ~ 8500 and 8900 A) is provided in Cenarro et al. (2001, 2006) on the basis of an extensive, 
empirical stellar library in the near-infrared. 

We have measured the TiO bands at A ~ 7100, 7600, and 8500 A using the different 
index definitions listed in Table 3. TiOyioo is a classic atomic index for the TiO band at 
A ~ 7100 A that consists of two continuum bandpasses and one central bandpass. For the 
TiO band at A ~ 8500 A we have followed the generic 12 slope-like index definition of Cenarro 
et al. (2003). In short, the signal of pixels within five continuum bandpasses (see Table 3) 
are used to fit a straight line — a pseudo-continuum, Fc(X) — over the region dominated by 
the TiO bands. The Ti0 8 5oo index is thus computed as the ratio between the pseudo- 
continuum values at the central wavelengths of the reddest and bluest continuum bandpasses: 
TiOg5oo = i 7 b(A8784.0)/Fc(A8479.0). In a sense, it may be considered to be a measure of the 
local pseudo-continuum slope, so TiOgsoo values around 1 mean that no TiO band is present 
in the spectra. Finally, the strength of the TiO band at A ~ 7600 A, Ti0 7 6oo, is defined 
to be the mean of three normalized, different index definitions: Ti0 76 oo(l), Ti07 6 oo(2), 
and Ti0 760 o(3). Ti0 760 o(l) and Ti0 760 o(2) are classic atomic index definitions — the latter 
having a narrower central bandpass — whereas Ti0 760 o(3) [= F C (A7734.5)/F C (A8099.5)] is 
another slope-like index defined by means of six continuum bandpasses (see Table 3). Since 
the spectra of our target stars are usually affected by inter-CCD gaps located around this 
spectral region, we preferred to employ different index definitions in order to ensure a large 
number of training set stars having available index measurements for this TiO band. Thus, 
in case any of the above index definitions was not reliable or impossible to measure, means 
of the remaining reliable measurements were computed to derive the final Ti0 7 6oo value. 

Two-dimensional RGB and dwarf PDFs for each of the three TiO diagnostics were 
constructed following the same procedure as for the Nai line (§3.2.4). Figure 8 shows the 
iso-probability contours for the dwarf and RGB PDFs, along with the position of the training 
set stars overlaid on the contours. It is clear that TiO diagnostics are not really useful for 
(V — I)o colors < 2.0 (since no apparent TiO bands are found for those temperatures). 
However, the TiO strengths of dwarf and RGB stars follow two different increasing trends 



12 Generic indices, denned as those having multiple continuum and central bandpasses, are specially suited 
to measure spectral features in regions crowded by other contaminating features, sky emission line residuals, 
and telluric absorption. 
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with increasing (V — 1) . Since Ti0 760 o might be affected by nearby telluric absorption, the 
discriminating power of this index is not as clear cut as it is for Ti0 7 i o an d Ti0 85 oo- 



3.3. Computation of Likelihood Values 

For each star in our survey with a measured blueshift, the probability the star is a red 
giant (-Pgiant) or a dwarf (-Pdwarf) is calculated based on the normalized RGB and dwarf star 
PDFs. The likelihood a star i is a red giant in a given diagnostic j is computed using the 
formula 

P ■ \ 

1 giant \ 



^ = l°g -= • (6) 

\ dwarf / 

To determine a star's overall likelihood value, we compute a weighted average of all the 
individual likelihoods available for each star: 

(Ih) = ^ , (7) 



E 



summed over the available diagnostics. In general, all available individual likelihoods for 
a given star receive equal weight, Wj = 1. The only exception to this is for stars that are 
outliers in both the RGB and dwarf PDFs in any of two-dimensional diagnostics (Na i EW, 
CMD position, and [Fe/H] pho t vs. [Fe/H] spcc ). For such stars the corresponding diagnostic is 
assigned a lower weight (wj < 1) as described below. The purpose of this is to downweight, 
and in extreme cases effectively remove from the overall likelihood estimate, diagnostics in 
which the star lies in a region of parameter space that is poorly sampled by both the RGB 
and dwarf star training sets. For the same reason, the individual likelihoods are capped at 
±5, which corresponds to a P g i an t to Pdwarf ratio of 10 5 or 1CT 5 . 

Specifically, this downweighting occurs for stars that satisfy the criteria: Pgiant < 3<7 gi ant 
and Pdwarf < 3o"dwarf m a given diagnostic, where the probability thresholds cr g ia n t and a^warf 
represent the probability levels which include 90% of the RGB and dwarf training set stars, 
respectively. The weight Wj assigned to such a star in the j-th diagnostic is: 

\PJ giant ^ \PJ dwarf 

where w is set to 2/9 to ensure that the weights are well-behaved for stars that are close to 
the above thresholds. We also investigated weighting the individual likelihoods for all stars 



-20- 



in all diagnostics based on the measurement errors in the relevant parameters (§2.3), but 
found no significant differences in the overall likelihood distributions. 

We found that the addition of the K i and TiO diagnostics did not improve our ability 
to separate RGB and dwarf stars, and in fact hurt it by putting too much weight on (V — 1) 
colors. There is strong covariance among the Nai, Ki, and TiO diagnostics because they 
share a common color axis (see § 4.2.1 for more details). The three TiO diagnostics are not as 
sensitive as the Na i diagnostic — the vertical separation between the dwarf and RGB PDFs 
in the TiO diagnostics is relatively small (Fig. 8) — and the degeneracy between the RGB 
and dwarf PDFs for hot stars in the K i diagnostics extends to redder colors than in the Na i 
diagnostic (compare Figs. 4 and 7). Furthermore, the fractional measurement error in the 
E W is larger for K i and TiO than for Na i: the K i lines and Ti0 760 o band are affected by the 
atmospheric A-band, and the two reddest TiO bands are occasionally affected by spectral 
continuum discontinuity artifacts at the inter-CCD interface in DEIMOS. 

For these reasons we do not include the K i and TiO diagnostics in our final calculation 
of the overall likelihood. We have nevertheless decided to present the details of the five K i 
and TiO diagnostics as they may be useful for other data sets. Moreover, in § 4.3.1 we discuss 
future modifications that should render the Nai, Ki, and TiO diagnostics independent of 
one another. 



3.4. Overall Likelihood Distributions 

Figure 9 shows the distribution of overall likelihood values for each of our Keck/DEIMOS 
fields. These histograms use all available diagnostics (excluding K i and TiO) for each star. 

The overall likelihood distributions for each field were used to determine whether a 
star was a red giant or a dwarf. The overall likelihood ({Li)) histograms in Figure 9 show 
obvious peaks at > +1 (RGB) and < +1 (dwarf). The relative height of the peaks change 
with radius: in fields with approximately equal numbers of RGB and dwarf stars we see a 
clear bimodality (ie. field al3), while for inner fields we see a strong peak of RGB stars with 
a tail of dwarf stars, and vice versa for the outer fields. Stars with (Li) > are designated 
M31 RGB stars, and stars with (Lj) < are designated foreground MW dwarf stars. 

Figure 10 illustrates the power of our RGB/dwarf separation technique using stars in the 
R = 30 kpc minor- axis field aO. The individual likelihoods — defined as Lj = (P g iant/-Pdwarf) 
for the j-th diagnostic — for the five primary diagnostics are plotted against one another (Lj 
vs. Lfc). Histograms for each individual likelihood Lj are plotted in the panels along the 
lower-right diagonal. It is encouraging to see that stars that are classified as M31 RGB stars 
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by our likelihood-based method (blue points) tend to lie in the top right quadrant of all 
panels (Lj, L k > 0) while those classified as MW dwarf stars (red points) tend to lie in the 
bottom left quadrant (Lj, < 0). No single diagnostic is able to discriminate perfectly 
between M31 RGB and MW dwarf stars, however the combination of diagnostics is very 
effective at separating the two stellar types. 

Although there is no explicit weighting of the diagnostics (§ 3.3), some are more powerful 
than others. Diagnostics with a large separation between the RGB and dwarf PDFs (relative 
to the widths of the PDFs) will have a large range of individual likelihood values and vice 
versa (e.g., compare the range of L v vs. Lcmd values in Fig. 10). Thus, diagnostics with a 
small separation between the dwarf and RGB PDFs will implicitly have less of an effect on 
the overall likelihood value. 

The overall range of Lcmd and I/ddosi is small with respect to the other diagnostics. 
This is because of the substantial overlap between the RGB and dwarf PDFs for these 
two diagnostics. Despite the small range, there appears to be a fairly clear RGB/dwarf 
separation in both of these diagnostics, indicating that our measurement error in these 
quantities is small compared to the width of the corresponding distribution. In field aO 
the Lpe/H diagnostic appears to be the least discriminatory of all the diagnostics. This is 
because of instrumental/data reduction problems with our early DEIMOS spectra (most of 
our aO spectra are from fall 2004). The Can triplet in particular is severely affected by sky 
subtraction errors in the aO spectra. By contrast the Na i doublet is in a part of the spectrum 
free of strong night sky emission lines and is well removed from the DEIMOS inter-CCD gap 
so the Z^Na diagnostic is well behaved in field aO. Despite its relative poor performance in 
field aO, the L Fe / H diagnostic in general works better than the CMD diagnostic and as well 
as the DD051 diagnostic. 

We next attempt to quantify how each individual diagnostic performs on the secure 
samples of M31 RGB stars and MW dwarf stars (Class +3/+2 and —3/— 2, respectively; 
see §3.5). Depending on whether the individual likelihood value for the j-th diagnostic and 
i-ih secure M31 RGB star [as defined by Eqn. (6)] is > +0.5, between —0.5 and +0.5, or 
< —0.5, that star is deemed to have a correct, uncertain, or incorrect classification. Similar 
classifications are attempted for secure MW dwarf stars using individual diagnostics. The 
percentages of correct, uncertain, and incorrect classifications are listed in Table 4 for each of 
the five primary diagnostics using secure RGB/dwarf stars in two broad radial bins: R = 30- 
60 kpc (fields aO, a3, and al3) and R = 60-165 kpc (fields al9, m6, bl5, m8, and mil). The 
radial velocity diagnostic is the most effective (highest percentage of correct classifications; 
the lowest percentage of uncertain/incorrect classifications) while the CMD diagnostic is the 
least effective. The Nai EW diagnostic is the second best; the DD051 and [Fe/H] pho t vs. 
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Table 4: Comparing the effectiveness of the five primary diagnostics. 



Diagnostic(s) R = 30-60 kpc R = 60-165 kpc 

Secure RGB (142) Secure Dwarf (70) Secure RGB (22) Secure Dwarf (202) 





Corr. 


Unc. 


Inc. 


Corr. 


Unc. 


Inc. 


Corr. 


Unc. 


Inc. 


Corr. 


Unc. 


Inc. 


L v 


96.5 


2.8 


0.7 


85.7 


11.4 


2.9 


90.9 


4.5 


4.5 


92.6 


5.0 


2.5 


L-DD051 


70.4 


26.1 


3.5 


64.3 


20.0 


15.7 


68.2 


27.3 


4.5 


73.3 


20.3 


6.4 


-^Na 


80.3 


16.9 


2.8 


68.6 


25.7 


5.7 


77.3 


22.7 


0.0 


71.3 


21.3 


7.4 


£CMD 


44.4 


54.9 


0.7 


21.4 


72.9 


5.7 


63.6 


36.4 


0.0 


24.8 


73.3 


2.0 


Lfc/K 


66.2 


23.9 


9.9 


85.7 


11.4 


2.9 


54.5 


27.3 


18.2 


79.2 


17.8 


3.0 


(Lh h 


94.4 


4.9 


0.7 


84.3 


14.3 


1.4 


90.9 


9.1 


0.0 


90.1 


7.4 


2.5 


(L)s c 


95.8 


4.2 


0.0 


94.3 


4.3 


1.4 


90.9 


9.1 


0.0 


95.5 


3.0 


1.5 




100.0 


0.0 


0.0 


94.3 


4.3 


1.4 


100.0 


0.0 


0.0 


96.5 


2.0 


1.5 



Percentage of secure RGB /dwarf stars in a given radial bin that were correctly classified by a given diag- 
nostic/combination of diagnostics ("Corr."), had an uncertain classification ("Unc"), or were incorrectly 
classified ("Inc."); see text for details. Secure RGB stars are defined to be Class +3 and +2 stars while 
secure dwarf stars are defined to be Class —3 and —2 stars (see § 3.5 and Table 5); the total number in each 
radial bin is given in parentheses after the column headings. 

b Weighted average of the two best diagnostics: L v and L^ a . 

c Weighted average of the three best diagnostics: L v , L^ a , and Xddosi- 

^Weighted average of the four best diagnostics: L v , L^ a , Lddosi, and ^Fo/h- 

[Fe/H] spcc diagnostics are comparable. Weighted averages of various subsets of diagnostics 
are also tested; their percentages are listed next in Table 4. 

3.5. Subclassification of the Red Giant and Dwarf Star Samples 

In general, stars with (Li) > are M31 RGB stars while those with (Li) < are 
foreground MW dwarf stars. We have developed a classification scheme which divides each 
of the RGB and dwarf samples into three classes, with Class +3 (—3) being the most secure 
and Class +1 (—1) the least secure. The classes are based primarily on (Li), but Xcmd is 
used as a secondary criterion (§ 3.2.5). The most secure RGB (Class +3) and dwarf stars 
(Class —3) are identified as those with (Li) > 0.5 and (Li) < —0.5, respectively. However, 
to be classified as a Class +3 object, a star must also pass the X CMD > 0.0 criterion, which 
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ensures that it falls within the color range of theoretical RGB isochrones at the distance of 
M31. 

Stars with (Lj) > 0.5 and —0.05 < X CMD < are designated Class +2. The errors in 
our X CM D measurement are approximately 0.05 (the errors are a little larger than this at 
the faint end of our sample and smaller at the bright end). Thus, Class +2 stars are those 
with large (Li) values that lie slightly bluer than (but roughly consistent with) the most 
metal-poor isochrone. 

Table 5: Confirmed M31 RGB and MW dwarf stars for each of the fields. a 



Field 


R 


No. Sci. 


No. M31 RGB Stars 


No. MW Dwarf Stars 


Comments 




(kpc) 


Spectra b 


Class 


Class 


Class 


Class 


Class 


Class 










+3 


+2 


+ 1 


-3 


-2 


-1 




Hll 


12 


234 


102 





13 


17 


1 


6 




H13s 


21 


221 


83 


1 


20 


18 


1 


6 


giant southern stream field 


H13d c 


25 


136 


67 






19 






disk field 


aO 


30 


245 


61 


4 


10 


30 


2 


3 




a3 


33 


224 


59 





8 


13 





4 


giant southern stream field 


al3 


60 


141 


16 


2 


2 


23 


2 


3 




al9 


81 


68 


4 





2 


23 





3 




m6 


87 


138 


7 


2 





44 


3 


8 




bl5 


95 


127 


5 





2 


24 


1 


10 




m8 


121 


109 


1 





2 


22 


2 


5 




mil 


165 


283 


3 





4 


82 


1 


11 





a The subclassification of the M31 RGB stars into Class +3, +2, and +1 and MW dwarf stars into Class -3, 
—2, and —1 (very secure, secure, and marginal) is discussed in §3.5. 

b Number of unique science spectra, excluding duplicate measurements (§2.3), Q = —2 instrumental failures 

(§2.2), and alignment stars. 
c The photometry in our H13d field is currently undergoing recalibration so we do not attempt to subclassify 

the M31 RGB stars and MW dwarf stars at this point. 

The least secure/marginal M31 RGB stars (Class +1) are defined to be those with 
0.0 < (L^ < 0.5 and Xqmd > —0.05. Likewise, the marginal MW dwarf stars (Class — 1) 
are those with —0.5 > (Li) < 0.0 and Xcmd > —0.05. We expect some misclassification of 
stars within the range —0.5 < (Li) < 0.5, because stars with (Li) values in this range often 
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have a spread of individual likelihood values, some positive (they follow the RGB PDF) and 
some negative (they follow the dwarf PDF). 

Finally, stars with (Lj) > —0.5 and X CMD < —0.05 are designated Class —2 objects. 
These objects fall considerably blueward of the most metal-poor theoretical isochrone in the 
CMD. We disregard the (Li) values of these stars because the diagnostic method does not 
work well for such blue objects (§§ 4.1.2-4.1.3). 

This classification scheme allows us to systematically identify marginal stars and remove 
them from our analysis. Table 5 lists the number of M31 RGB and MW dwarf stars per 
class in each field. Figure 11 (top and middle) shows the distribution in the CMD of the 
three RGB classes and three dwarf classes, respectively. 

4. Discussion of the Method 
4.1. Statistical Tests 

4-1.1. Evidence of M31 RGB Stars in the Outermost Fields 

Is it possible that the stars identified as secure M31 RGB stars in our outermost fields 
are actually misidentified MW dwarf stars (e.g., stars in tails/outskirts of the dwarf star 
distributions in the various diagnostic parameter spaces)? We do not know a priori what 
fraction of the overall dwarf population is in the form of such outliers, but expect the fraction 
to be the same in all fields. By contrast, the ratio of secure RGB stars to secure MW dwarfs 
is observe to decline monotonically with increasing projected distance from M31's center 
(Guhathakurta et al. 2005). 

Given the very small number of secure M31 RGB stars found in our outermost halo 
fields, it is very important to check whether the properties of these stars are similar to 
those of the bulk of M31 RGB stars in our sample. Figure 12 compares the distribution of 
very secure M31 RGB (Class +3) stars in our two outermost fields m8 and mil to that of 
foreground MW dwarf stars in four of the diagnostic plots. The handful of very secure M31 
RGB stars and the far more numerous secure M31 dwarf stars follow the RGB and dwarf 
PDFs, respectively. This suggests that the Class +3 objects found in these fields are indeed 
genuine M31 RGB stars. 

Figure 13 shows the (V — I)o color distribution of secure+marginal M31 RGB (Class 
> 0) and secure+marginal MW dwarf (Class < 0) stars. The color distribution of the M31 
RGB stars peaks at (V — I) ~ 1.5 with a tail to redder colors. The MW dwarf stars have 
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a bimodal distribution, with a peak at (V — I)o ~ 2.3 and a smaller peak at (V — I)o ~ 1. 
We fit the color distribution of stars in each of three radial bins as a linear combination 
of the RGB and dwarf color distributions: the R = 30-85 kpc, R = 85-125 kpc, and 
R = 165 kpc (mil) samples are best fit by 75%/25%, 45%/55%, and 5%/95% RGB/dwarf 
combinations, respectively. Stars in our furthest halo field mil closely follow the MW dwarf 
color distribution, except for a slight bump at (V — I)o ~ 1.5. Such a bump is not present in 
the dwarf color distribution, but the M31 RGB color distribution has a peak at about this 
color. This bump in the mil color distribution is where the Class +3 and all but one of the 
Class +1 stars are located. The colors of these objects suggest that they are in fact M31 
RGB stars. 



4-1.2. Nature of the Bluest Stars (Class ±2) 

There are 49 stars in our sample with X C md < (i.e., bluer than the most metal-poor 
isochrone, (V — 1) < 1). The Nai, Ki, and TiO diagnostics have no discriminating power for 
such blue stars. Also, these stars lie in the remote outskirts of both RGB and dwarf PDFs 
in the CMD and [Fe/H] diagnostics; the downweighting of outliers in the overall likelihood 
calculation ensures that these two diagnostics carry little weight. This leaves only two out 
of the five diagnostics, L v and Lddosi- As a further complication, some of the bluest MW 
dwarfs in our sample also have very negative velocities that overlap the distribution of M31 
RGB stars (see §4.2.3 and Fig. 18). For these reasons, we have taken a close look at these 
blue stars. 

Metal-poor isochrones crowd together in the CMD — e.g., an isochrone with [Fe/H] = —3 
is only slightly bluer than the bluest ([Fe/H] = —2.3) isochrone in Figure 11 (Vandenberg 
et al. 2006). Thus any star that lies well to the blue of our most metal-poor isochrone 
is almost certainly a MW dwarf star. A comparison of the (V — I) color distributions 
of RGB and dwarf stars (thin histograms in middle panel of Fig. 13) confirms that stars 
with (V - J)o< 1 are « lOx more likely to be MW dwarfs than M31 RGB stars. This is 
reinforced in Figures 7 and 8: the dwarf PDF contours based on the final training set extend 
to (V — I) < 1 whereas the RGB contours cut off sharply at (V — I) ~ I. 

Of the 49 blue stars, 26 have overall likelihood (Lj) < —0.5 and we feel confident in 
designating them Class —3 (very secure MW dwarf stars). Eleven of the 49 have over- 
all likelihood (Li) > —0.5 but are significantly bluer than the most metal-poor isochrone 
(Acmd < —0.05); we designate them Class —2 (secure MW dwarf stars). There are another 
11 stars with overall likelihood (Li) > +0.5 that lie slightly blueward of the most metal-poor 
isochrone (0.05 < A CMD < 0); they are designated Class +2 (secure M31 RGB stars). The 
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full set of classification criteria are given in §3.5. 

Figure 14 shows the distribution of Class +2 and —2 stars in the velocity and DD051 
diagnostic plots. The distribution of Class +2 stars in both diagnostics indicates that they 
are far more likely to be drawn from the RGB PDF than the dwarf PDF. The case for 
the Class —2 stars is more complicated. Their DD051 parameters span a broad range 
[Fig. 14(d)], but this is to be expected: the locus of blue ((V — I)q < 1) dwarf stars lies 
close to the RGB selection box in the (M — DD051) vs. (M — T 2 ) color-color plot (see 
Fig. 5 of Palma et al. 2003) so they are sometimes assigned a large DD051 parameter 
value. The velocities of Class —2 stars are in the overlap region of the RGB/dwarf PDFs 
[Fig. 14(c)]. The Galactic star count model (Bahcall & Soneira 1984; Ratnatunga & Bahcall 
1985) predicts that the radial velocity distribution of blue MW dwarf stars in our sample 
should be skewed toward negative values compared to that of faint red MW dwarfs (see Fig. 8 
of Reitzel & Guhathakurta 2002). This is because MW dwarfs with (V — I) < 1 must be 
main-sequence turnoff stars in the MW halo at heliocentric distances of 15-50 kpc (absolute 
mag: (M/) MSTO ~ 4.0; apparent mag range of our sample: 20 < Iq < 22.5). Members of the 
hot, non-rotating MW halo are expected to span a broad range of radial velocities centered 
on —175 km s _1 (reflex of solar motion in the direction of M31 — Reitzel & Guhathakurta 
2002), consistent with the observed distribution of Class —2 stars. 

Even though Class +2 and —2 stars fall far outside the RGB and dwarf PDFs in the 
[Fe/H] diagnostic, it is instructive to compare their [Fe/H] p i lot and [Fe/H] spcc values. Class +2 
stars have median [Fe/H] p h t and [Fe/H] spec of —2.5 and —1.9, respectively, which are in 
reasonable agreement given the large measurement errors for metal-poor stars. By contrast, 
Class —2 stars have median [Fe/H] p hot and [Fe/H] spcc of < —4 and —2.0, respectively (the 
[Fe/H] phot estimates are admittedly based on gross extrapolation). 

In conclusion, the velocities, DD051 parameters, and (with greater uncertainty) [Fe/H] 
estimates of the Class +2 and —2 stars are consistent with their designations as secure M31 
RGB and MW dwarf stars, respectively. 

4-1.3. Nature of the Marginal Cases (Class ±1) 

The (L^ distributions of RGB and dwarf stars are broad (Fig. 9): it is likely that some 
legitimate M31 RGB stars have scattered to negative (Lj) values, and some legitimate MW 
dwarf stars have scattered to positive (Lj) values. For this reason stars with \ (Li)\ < 0.5 are 
designated Class +1 or —1 (marginal RGB or dwarf stars; §3.5). 

If the shapes of the RGB and dwarf (Lj) distributions are the same from field to field, 
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the number of marginal RGB stars (Class +1) in a given field can be expressed as the sum 
of a fraction a of the number of secure RGB stars (Class +3 and +2) and a fraction b of the 
number of secure dwarf stars (Class —3 and —2) in that field: 

iV(Class +1) = a[iV(Class +3) + iV(Class +2)] + b [iV(Class -3) + iV(Class -2)]. (9) 

Similarly the number of marginal dwarf stars (Class —1) can be expressed as: 

7V(Class -1) = c[iV(Class +3) + iV(Class +2)] + d [iV(Class -3) + iV(Class -2)]. (10) 

Figure 15 (top) shows the ratio of marginal M31 RGB (Class +1) stars to secure RGB 
and dwarf stars (Class +3, +2, —3, and —2) as a function of the ratio of secure RGB to 
dwarf stars. Each data point represents one of our fields and the abcissa values are roughly 
in order of decreasing projected radial distance of the field from the center of M31. The data 
are consistent with a = 0.07 and b = 0.03 (solid line) but can also be fit by a = b = 0.05 
(dashed horizontal line). The marginal dwarf (Class —1) stars are well fit by c = 0.03 and 
d — 0.15 [solid line in Fig. 15 (bottom)]. 

Based on the first pair of (a, b) values, about 75% of the stars in our overall Class +1 
sample are expected to be true M31 RGB stars while the remaining 25% are expected to be 
MW dwarf stars. The mix of Class —1 stars is expected to be 80% MW dwarfs and 20% 
M31 RGB stars. These percentages apply to our overall samples of marginal stars; the mix 
varies from field to field of course. 

Figure 16 shows the distribution of marginal M31 RGB (Class +1) stars in four of 
our five primary diagnostic plots (all except the CMD diagnostic). These stars have radial 
velocities in the region of overlap between the RGB and dwarf PDFs, a large spread of 
DD051 values, and (V — I)q colors mostly in the range 1-2.5 which places them in the 
region of overlap of the RGB and dwarf PDFs in the Nai diagnostic plot — all consistent 
with their marginal overall likelihood value. The distribution of these stars in the [Fe/H] 
diagnostic plot is consistent with most being M31 RGB stars. 

Figure 17 shows the distribution of marginal MW dwarf (Class —1) stars in the same 
four diagnostic plots. Their (V — 1) colors are mostly in the range 1-2, as for the Class +1 
cases. The Nai diagnostic plot is not conclusive but the radial velocity, DD051, and [Fe/H] 
diagnostic plots favor the MW dwarf designation. 

In summary, the Class ±1 stars are not as blue as the Class ±2 stars, but most of 
the arguments about the performance of our method on blue stars applies here as well, 
only to a lesser degree (§4.1.2). The statistical analysis presented above [Fig. 15; Eqns. (9- 
10)] indicates that most of the Class +1 objects should be M31 RGB stars and most of 
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the Class —1 objects should be MW dwarf stars. The distribution in the diagnostic plots 
support this hypothesis. 

4.2. Covariance among Diagnostics 

4.2.1. Na, K, and TiO 

The Nai, Ki and TiO diagnostics (Figs. 4, 7, and 8) are clearly not independent; not 
only do they share a common color axis, but a star's Nai, Ki and TiO line strengths all 
depend on both temperature (color) and surface gravity. The degeneracy between the dwarf 
and RGB PDFs in these diagnostics at bluer (V — I\ colors (especially in Ki and TiO) 
results in much of their discriminating power coming from the color distribution. Therefore, 
if most of the stars in the sample are bluer than (V — I) ~ 2.5, the inclusion of all five 
of these diagnostics (Nai, two Ki, and three TiO) will overly weight the color distribution 
in determining overall likelihood values. We choose to use only the Nai diagnostic, since 
it has the most discriminating power of the five. However, the Ki and TiO diagnostics do 
have some discriminating power (beyond the color distribution) for redder stars, and could 
be used in an appropriate sample. Finally, although the two Ki diagnostics are obviously 
not independent of each other (nor are the three TiO diagnostics), combining the two Ki 
and three TiO diagnostics is beneficial since this will reduce the scatter caused by individual 
line strength measurement errors. 

4.2.2. Na, CMD, and [Fe/Hj 

Although the vertical axes of the Nai, CMD, and [Fe/H] diagnostics (Figs. 4, 5, and 6) 
are independent of each other, there is a complicated dependence between the horizontal axes 
of these three diagnostics. The horizontal axis of the Nai diagnostic is (V—I)o color. Both the 
horizontal axes of the CMD and [Fe/H] diagnostics are dependent on color: X CMD depends 
on position within the CMD with relation to the theoretical RGB isochrones (§3.2.5), as 
does [Fe/H] phot (§3.2.6). However, the dwarf and RGB PDFs in both the CMD and [Fe/H] 
diagnostics cover the same horizontal range. All the discriminating power in these two 
diagnostics comes from the vertical axes, which are independent. 
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4-2.3. Correlations within the Dwarf Star Sample 

We have explored all secure M31 dwarf stars for possible correlations between radial 
velocity and (V — I)o color and/or DD051 parameter (Fig. 18). Most dwarf stars have small 
DD051 parameters, as expected. Moreover, regardless of radial velocity, there is a tail to the 
distribution of DD051 parameters reaching all the way up to values near unity. No strong 
trend is seen between radial velocity and (V — I) color either. The few dwarf stars with 
very negative radial velocities (vh e \ < —225 km s" 1 ) are all very blue. However, the rest 
of the dwarf stars display a wide range of (V — I)o colors and show no correlation between 
radial velocity and color. 

To ensure that the stars we have identified as secure M31 RGB stars are not simply 
the tail of the dwarf distribution in the outermost, sparsest M31 halo fields, we explored 
the dwarf training set for correlations between the positions of stars in the diagnostics. If 
a star in the tail of the dwarf distribution in one diagnostic is also in the tail of the dwarf 
distribution in the other diagnostics, it will receive a low probability of being a dwarf star 
in all of them, adding up to an spuriously high probability of being an RGB star. For 
four of the most powerful diagnostics (radial velocity, Nai EW vs. (V — I)o color, DD051 
parameter, and [Fe/H] p hot vs. [Fe/H] spcc ), we chose dwarf training set stars in the tail of the 
dwarf distribution and investigated where they fell in the other three diagnostics. Figure 19 
shows where stars selected to be in the tail of the dwarf distribution in the Nai diagnostic 
{{V — I)o < 2) fall in the other diagnostics. It is clear that they do not fall in the tail of the 
dwarf distribution in radial velocity, DD051 parameter, or in [Fe/H] spec vs. [Fe/H] p h t- The 
only correlation is a slight shift to lower [Fe/H] p h t values. This is expected, since [Fe/H] p h t 
is related to (V — I)q color. The tails of the dwarf distribution in the other diagnostics show 
the same behavior as in this example: no correlation was found between diagnostics. 



4.3. Shortcomings of the Present Method and Future Improvements 

While our likelihood-based RGB/dwarf separation method is vastly superior to the radial 
velocity cut used in other spectroscopic surveys of M31 stars (§4.4), it is far from optimal 
and will no doubt undergo improvements in the not-too-distant future. These improvements 
are beyond the scope of the present paper and will be tackled in a future paper; we list them 
here for the sake of completeness. 
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4-3.1. Color Dependence of the Diagnostics 

The covariance of the diagnostics has been discussed in §4.2. The degeneracy between 
the horizontal axes (color-dependence) of the two-dimensional diagnostics can be lifted by 
converting them to one-dimensional diagnostics. One method which holds promise for the 
future is to fit a spline to the distribution of training set stars in each diagnostic. The distri- 
bution of vertical distances of the training set stars from the fit will yield a one-dimensional 
PDF. Once a two-dimensional diagnostic is collapsed into a one-dimensional diagnostic in 
this way, the latter will have no sensitivity to differences in the (V — I)q color distribution 
between M31 RGB and MW dwarf star populations. This will enable us to incorporate the 
Ki and TiO diagnostics into the overall likelihood. 

4.3.2. MSl's Radial Metallicity Gradient 

Another future improvement to the method will be to account for radial trends in the 
M31 RGB star properties. We are currently using one M31 RGB training set for all of 
our fields. However, we have discovered that M31 RGB stars at large radial distances are 
systematically more metal-poor than those at smaller radial distances (Kalirai et al. 2006a). 
Our present method is sub-optimal in that our RGB PDFs do not track the change in the 
(V — I)o color distribution of M31 stars with radial distance and are based on a training set 
that is drawn mostly from M31's inner spheroid. In order to track this radial trend, we will 
need a significantly larger sample of RGB stars in M31's outer halo to use as a training set. 
We expect that a much larger outer halo RGB sample will be available in the next few years. 

4-3.3. Accounting for Measurement Error and Eliminating Analytic PDFs 

The likelihood-based method developed in this paper does not explicitly take advantage 
of our (admittedly rough) knowledge of the measurement errors in the quantities used in 
our various diagnostic plots (§2.3). For idealized error distributions, each training set star 
can be treated as a Gaussian whose width is the quadrature sum of its own measurement 
error and that of the star for which the likelihood estimate is being made. Naturally, these 
would be one-dimensional Gaussians for the radial velocity and DD051 diagnostics and two- 
dimensional Gaussians for the Nai, CMD, and [Fe/H] diagnostics. The integral under the 
Gaussian would be the same for each training set star. The M31 RGB and MW dwarf PDFs 
would then be the sum of all the Gaussians corresponding to the training set stars. The 
integral under each PDF would be set to unity. We hope to implement such a procedure 
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in the future; with enough stars in the training set the sum of all the Gaussians should be 
smooth enough that it will no longer be necessary to fit analytic functions in order to derive 
PDFs. 



4.3.4. Estimating the Number of M31 RGB Stars 

Through most of this paper we have focused on the question: Which stars are M31 RGB 
stars and which ones are MW dwarf stars? Alternatively, one might ask: What is our best 
statistical estimate of the number of M31 RGB stars and MW dwarf stars in the sample? 
The first question is clearly important for addressing a wide variety of scientific topics (such 
as the metallicity or dynamics of a clean sample of M31 RGB stars), but the second question 
is more relevant when it comes to studies of the spatial density distribution of RGB stars 
and surface brightness profile and M31's halo. 

The distinction between the above two questions can be illustrated by the following 
simple scenario. Let us assume there is a measurable x and that M31 RGB stars and MW 
dwarf stars are characterized by different, but overlapping x distributions. Armed with only 
this information, it is impossible to assign a clear designation to any star whose x value lies 
in the overlap region. However, if the shape of underlying parent x distribution is known 
for M31 RGB and MW dwarf stars, one can fit a weighted sum of the RGB and dwarf star 
parent distributions to the observed x distribution and thereby determine the fraction of 
each stellar type in the sample. 

In the context of our method, the RGB and dwarf PDFs for each diagnostic (modulo the 
other planned improvements of course) can be treated as the underlying parent distributions. 
One could then fit, simultaneously for all diagnostics, a weighted sum of the PDFs to the 
observed distribution of points. The outcome of the fit would be our best estimate for the 
M31 RGB fraction in each field. We plan to explore such a method in a future paper. 

4.4. The Diagnostic Method Versus a Radial Velocity Cut 

We assess the performance of the likelihood-based method presented in this paper by 
comparing it to the use of a simple radial velocity cut to select M31 RGB stars. Most 
previous spectroscopic studies of M31 RGB stars have resorted to using radial velocity cuts: 
for example, Reitzel & Guhathakurta (2002) used v < —220 km s" 1 while Ibata et al. (2005) 
used v < —100 km s _1 . 

To investigate the improvement in M31 RGB sample size gained through using our 
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diagnostic method, we applied a velocity cut of v < —220 km s _1 to our smooth spheroid 
sample. (The disk-dominated field H13d and the stream stars in fields H13s and a3 are 
excluded from this count.) This resulted in an RGB sample of 208 stars, compared to 207 
stars identified by the likelhood method as secure (Class +3 and +2) RGB stars. However, 
only 192 stars are in common between the two samples. This velocity cut recovers 93% of 
the M31 RGB stars, and 8% of the velocity-selected RGB stars are actually MW dwarf star 
contaminants. If we restrict our analysis to the outer halo fields (fields al3 through mil), 
this velocity cut yields 42 stars while our likelihood method yields 40 secure RGB stars, with 
36 stars in common. Thus, a velocity cut of v < —220 km s -1 in the outer halo recovers 
90% of the M31 RGB stars in these fields, but suffers from a MW dwarf star contamination 
rate of 19%. As an extreme case, an L v > +0.5 cut in the radial velocity diagnostic (which 
corresponds v < —190 km s -1 ) yields four stars in our outermost field mil (R = 165 kpc), 
but only two of these are secure M31 RGB stars, while one each is a secure MW dwarf and 
a marginal MW dwarf — i.e., a 50% contamination rate. Moreover, one of the secure M31 
RGB stars in this field fails this velocity cut. 

If a less conservative velocity cut is made, all the RGB stars can be recovered, but at 
the price of a larger dwarf contamination rate. When a velocity cut of v < —100 km s _1 is 
made in Hll, our innermost smooth spheroid field, it yields 120 stars, while our likelihood 
method finds 102 secure RGB stars. The sample selected solely on the basis of this velocity 
cut would have a dwarf contamination rate of 15%. If this velocity cut is made in field aO 
as well as Hll, it yields 203 stars, versus 167 secure RGB stars, for a dwarf contamination 
rate of 17.7%. Finally, if a somewhat more conservative velocity cut of v < —160 km s~ x 
(as in Chapman et al. 2006) is made in the smooth halo fields beyond R = 30 kpc from the 
nucleus (aO, al3 through mil) the total number of stars selected is 140, compared to 105 
secure RGB stars, for a dwarf contamination rate of 25%. 

In conclusion, we find that the radial velocity cuts that have been used to isolate M31 
RGB stars in other spectroscopic surveys (e.g., Reitzel & Guhathakurta 2002; Ibata et al. 
2005; Chapman et al. 2006) yield samples that contain a significant fraction of foreground 
MW dwarf star contaminants and/or are incomplete. While this may be acceptable for 
statistical studies of the inner regions of M31 where the surface density of RGB stars greatly 
exceeds that of foreground MW dwarf stars, the dwarf contamination presents a serious 
obstacle for studies of M31's sparse outer halo. As discussed above, using a conservative 
radial velocity cut (say —200 km s" 1 ) reduces the dwarf contamination rate but causes other 
complications. For example, any dynamical analysis of a radial velocity-selected sample 
of RGB stars would have to take the cut into account. Given the large spread of M31 
RGB radial velocities around v^ 1 = —300 km s" 1 (in its dynamically hot bulge and halo 
and fast rotating disk), a radial velocity cut of —200 km s _1 would lead to substantial 
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incompleteness/bias. By contrast, our likelihood-based method produces a clean unbiased 
sample of M31 RGB stars that can directly be used for dynamical (and other) studies. 

4.5. Comparison to Photometric Studies 

In this section we switch our attention from comparison with other spectroscopic stud- 
ies to comparisons with photometric studies of the surface brightness profile of M31. It has 
long been recognized that star count studies are far superior to integrated light measure- 
ments (Pritchet & van den Bergh 1987, 1988, 1994). Star count studies typically select M31 
RGB candidates based on location within the CMD and use control fields for statistical sub- 
traction of the "background" (i.e., foreground MW dwarf star and background field galaxy 
contaminants — Pritchet & van den Bergh 1994; Durrell et al. 2004; Irwin et al. 2005). How- 
ever, systematic errors in background subtraction (e.g., due to photometric and/or seeing 
variations) limit these techniques to regions of M31 where the surface density of RGB stars 
is comparable to or greater than the background level (fi v ~ 30 mag arcsec -2 ). 

A variety of photometric filtering techniques have been used to increase the contrast of 
M31 RGB stars with respect to contaminants. An example of this is the use of UBRI pho- 
tometry to reject background galaxies (Reitzel, Guhathakurta, & Gould 1998). As discussed 
in §2, Ostheimer (2002) used DD051 photometry to select M31 RGB candidates. This 
technique reduces the surface brightness level of the background to pi v > 31 mag arcsec -2 
(Guhathakurta et al. 2005). 

In contrast to the above methods, using a combination of photometric and spectroscopic 
diagnostics allows us to reliably pick out very sparse groups of M31 RGB stars by rejecting 
foreground MW dwarf star and background field galaxy contaminants. The three secure M31 
RGB stars identified in our outermost field mil constitute only a few percent of the overall 
sample of DD051-selected RGB candidates in that field; the rest are MW dwarf stars or 
galaxies. We use the ratio of secure M31 RGB stars to secure MW dwarf stars, in conjunction 
with a Galactic model (Bahcall & Soneira 1984; Ratnatunga & Bahcall 1985), to estimate 
M31's surface brightness profile (Guhathakurta et al. 2005). This allows us to map the outer 
halo of M31 down to an unprecedented surface brightness level of /iy ~ 35 mag arcsec -2 . 
Kalirai et al. (2006a) study the metallicity distribution of M31 RGB stars isolated by our 
diagnostic method and find that the stellar halo is significantly more metal poor than the 
bulge; this is in contrast to findings from previous photometric studies of a metal-rich stellar 
"halo" in M31 (e.g., Durrell et al. 2004; Irwin et al. 2005). 
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5. Conclusion 

As part of a large spectroscopic survey of the outer halo of M31, we have developed a 
method to isolate a clean sample of RGB stars in M31. This method uses five diagnostics 
to isolate RGB stars: radial velocity, DD051 photometry, the EW of the Nai absorption 
feature, CMD position, and a comparison of spectroscopic vs. photometric [Fe/H] estimates. 
We also explored diagnostics based on the EWs of the K 7665 and K 769 9 absorption features, 
and the strengths of the TiO bands at 7100, 7600, and 8500 A. The first five diagnostics 
are more or less independent of one another; the Ki Nai, and TiO diagnostics all show a 
dependence on stellar effective temperature and are only effective for cool (red) stars. The 
overall likelihood based on a combination of diagnostics proves to be a much more powerful 
tool than any of the individual diagnostics alone. 

The diagnostic method gives us the ability to confidently identify very sparse popula- 
tions of M31 RGB stars, and has led to the discovery of stars as distant as R ~ 165 kpc (in 
projection) from the galaxy's center. This technique has allowed us to reach surface bright- 
ness levels of nv ~ 35 mag arcsec -2 and has enabled us to find an extended stellar halo in 
M31 that follows an R~ 2 6 surface brightness profile out to R ~ 165 kpc (Guhathakurta et al. 
2005). An investigation of the properties of these RGB stars shows that the stars become 
increasingly metal-poor with increasing distance from M31's center: beyond R > 60 kpc, the 
mean metallicity of our secure M31 RGB sample is found to be ([Fe/H] phot ) = —1.26 ± 0.10 
(Kalirai et al. 2006a). In addition to identifying sparse populations, our method for isolating 
M31 RGB stars has the benefit of permitting detailed dynamical studies of the M31 halo, 
since the selection of stars does not depend on a radial velocity cut and the resulting RGB 
sample is therefore not kinematically biased. The application of the diagnostic method to 
our M31 halo survey provides an unprecedented level of sensitivity, allowing us to reliably 
study the extent, structure, and kinematics of the remote M31 halo. 
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Fig. 1.— Sky position (J2000.0 RA/DEC) of the fields observed with DEIMOS spectroscopic 
slitmasks (solid circles). The small gridded squares indicate the full area (35' x 35') of 
the fields from the KPNO 4-m/Mosaic imaging survey of Ostheimer (2002). The resulting 
photometric DD051-, M-, and T 2 -band catalog was used to select spectroscopic targets for 
the DEIMOS slitmasks, each of which covers an area of roughly 16' x 4'. The majority 
of the fields in our DD051 survey lie close to the south-eastern minor axis of M31 and 
extend out to a projected distance from M31 of R = 163 kpc (field mil). The large bold 
open squares represent two of our CFHT MegaCam g'- and i'-band pointings, used to select 
spectroscopic targets for fields Hll and H13s. The pair of thin solid diagonal lines indicates 
the approximate extent of the visible disk of M31. The location of M33 is marked near the 
lower left. The concentric circles correspond to projected distances from M31's center of 30, 
60, 90, 120, and 165 kpc. A few lines of Galactic latitude are also marked. 
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Fig. 2. — Probability distribution functions for radial velocity in differential (top) and cu- 
mulative (bottom) form. The thin-lined histograms show the dwarf star training set; the 
dashed curves show the double Gaussian fit used to define the PDF. Since the red giant 
branch training set is selected on the basis of radial velocity, it cannot be used to define the 
radial velocity PDF. We instead adopt a Gaussian as the RGB PDF (dot-dashed curve), 
centered on M31's systemic velocity of —300 km s _1 with a width of a = 85 km s -1 , based 
on a fit to the inner spheroid field Hll sample (bold solid histograms). Dwarf stars lie close 
to fhei ~ km s" 1 but have a negative tail due to the reflex of the component of the Sun's 
velocity towards M31. As such, there is substantial overlap between the M31 RGB and MW 
dwarf radial velocity distributions. 
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Fig. 3. — Same as Figure 2 for the DD051 parameter. The bold solid histograms represent 
the RGB training set, a compilation of RGB stars from several fields close to M31's center. 
The DD051 parameter is assigned based on the position of a star in the (M — DD051) 
vs. (M — T 2 ) color-color diagram (Majewski et al. 2000): a value close to zero implies it is 
likely to be a dwarf star whereas a value close to unity implies it is likely to be an RGB 
star. The photometry is from KPNO 4-m Mosaic images in the Washington system M and 
T 2 bands and in the intermediate-width DD051 band, which includes the surface-gravity 
sensitive Mgb and MgH stellar absorption features at ~ 5100 A. Exponential fits to the 
dwarf and RGB training sets are used to define the PDFs. As expected, the MW dwarf 
training set distribution peaks at zero and that of the RGB training set peaks at unity, but 
there is significant overlap between the two distributions. 
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Fig. 4. — Two-dimensional PDFs for the equivalent width of the surface-gravity- sensitive Nai 
absorption line doublet at 8190 A as a function of dereddened (V—I)q color (which is sensitive 
to stellar effective temperature). The left panels show iso-probability contours for the RGB 
(top) and dwarf (bottom) PDFs, which are analytic fits to the distribution of training set 
stars (open circles). Both sets of PDF contours are shown in the right panels with the RGB 
and dwarf contours in bold in the top and bottom panels, respectively. The percentage of 
training set stars enclosed within each contour is indicated in the right panels (ordered from 
innermost to outermost); the percentage of analytic PDF space enclosed by the contour and 
the Poisson error are listed within parentheses. Cool dwarf stars with (V — I)q > 2 exhibit 
strong Nai absorption, while RGB and hotter dwarf stars show negligible Nai absorption. 
The color distribution of M31 RGB training set stars is asymmetric; it peaks at (V — I)q > 1 
but has a tail extending out to redder colors. The MW dwarf training set stars have a 
somewhat broader and symmetric color distribution centered on (V — 7)o~ 2.5. The Nai 
diagnostic is very effective at separating M31 RGB stars from MW dwarfs for stars redder 
than (V — I) ~ 2.5, but cannot distinguish between the two stellar types for (V — I) < 2. 
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Fig. 5. — Same as Figure 4 for position within the (I, V — I) color-magnitude diagram. The 
abscissa is the fractional distance across the isochrones: Xcmd — at the most metal-poor 
isochrone, [Fe/H] = —2.6, and Xcmd = 1 at the most metal-rich isochrone, [Fe/H] = +0.5; 
the ordinate is the fractional distance along the RGB isochrones: Fcmd = at I = 22.5, 
at the faint end of our spectroscopic sample, and Fcmd = 1 at the tip of the RGB. These 
parameters are calculated using theoretical RGB isochrones from Vandenberg et al. (2006), 
for an age of t — 12 Gyr and chemical composition of [ct/Fe] = 0, shifted to M31's distance 
modulus of 24.47 mag. Since M31 RGB stars tend to occupy the region of the CMD that 
is bounded by the isochrones (widest color spread near the tip of the RGB, progressively 
smaller color spread toward fainter I magnitudes), they form a roughly rectangular locus 
in (Xcmd, Fcmd) space. On average, foreground dwarf stars tend to be brighter in Iq than 
M31 RGB stars. Dwarfs are broadly distributed in the (/, V — I) CMD, but since the RGB 
isochrones curve toward redder colors as metallicity increases and tend to be more widely 
separated toward the tip of the RGB, dwarf stars at higher Fcmd values tend to have a lower 
mean Xcmd and smaller spread in Xcmd- 
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Fig. 6. — Same as Figure 4 for photometric versus spectroscopic metallicity estimates. The 
[Fe/H]phot estimate is derived by comparing the CMD position of each star to 12.6 Gyr model 
isochrones with [a/Fe] = spanning a wide range of metallicities (Vandenberg et al. 2006). 
The [Fe/H] spec estimate is derived from the equivalent width of the Can triplet absorption 
feature at ~ 8500 A, using empirical calibration relations based on Galactic globular cluster 
red giants (Rutledge et al. 1997a,b). M31 RGB stars show reasonable agreement between 
[Fe/H] phot and [Fe/H] spec , as expected. The [Fe/H] phot estimate for dwarf stars is obviously 
incorrect, as it is based on RGB isochrones at the distance of M31; dwarf stars tend to have 
relatively weak Can lines and generally lie well below the [Fe/H] p h t = [Fe/H] spec relation 
(diagonal lines in left panels). 
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Fig. 7. — Similar to Figure 4 for the Ki 7665 A (top) and 7699 A (bottom) lines. Data points 
show M31 RGB (left) and M31 dwarf (right) stars from the final training set overlaid on both 
sets of PDF contours, with the corresponding set of contours in bold. There is significant 
overlap between the M31 RGB and MW dwarf distributions for bluer colors [(V — 1) > 2.5]. 
However, the strength of the Ki absorption lines increases with increasing (V — I)o for dwarf 
stars, leading to a clear differentiation between the two stellar types at redder colors. 
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Fig. 8. — Same as Figure 7 for the TiO 7100, 7600, and 8500 A bands (top ^ bottom) , with the 
RGB and dwarf training set stars and PDFs shown in the left and right panels, respectively. 
There is substantial overlap between the RGB and dwarf PDFs, especially for the TiO 7600 A 
band for which the measurements are complicated by the presence of the telluric A band. 
No TiO bands are present for relatively hot stars with (V — I) < 2.0, so the diagnostics 
provide no RGB/dwarf discrimination in this regime. However, the TiO strengths of dwarf 
and RGB stars follow two slightly different increasing trends with increasing (V — I) , so 
that these diagnostics have some power for stars redder than (V — I) > 2.0. 
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Fig. 9. — Histograms of overall likelihood {{Li)) values, where Li = log(P g i ant /Pdwarf) for 
each diagnostic, for stars in the M31 outer halo fields. The fields are arranged in order 
of increasing projected distance from M31 (see Fig. 1). Confirmed RGB stars are those 
with {L^ > (shaded histograms) while (Li) < indicates dwarf stars (open histograms). 
There are clear peaks at < — 1 and > +1, indicating RGB and dwarf star distributions 
respectively: in fields with approximately equal numbers of RGB and dwarf stars, there is a 
clear bimodality (e.g., fields aO and al3). In inner fields there is a clear RGB peak with a tail 
of dwarf stars, while in outer fields there is a clear dwarf peak with a tail of RGB stars. As 
expected, the red giant fraction decreases with increasing distance from M31. The exception 
is field a3, which due to the increased RGB density in the southern stream has significantly 
fewer dwarf stars. We find secure RGB stars all the way out to field mil, which has three 
distinct outliers (M31 RGB stars) from the main distribution (Galactic dwarf stars). 
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Fig. 10. — The position of all stars in field aO at R = 30 kpc in probability ratio space for 
the five primary diagnostics. The panels along the right edge diagonal show histograms of 
Lj = log(Pg iant /Pdwarf) for each diagnostic j; positive values indicate the star is more likely 
to be an RGB star than a dwarf (in terms of that diagnostic) and vice versa for negative 
values. The blue and red symbols indicate M31 RGB stars and MW dwarfs, respectively, 
as determined by our likelihood-based method (§3.5) and are subdivided into: Class ±3 
(very secure: filled symbols), Class ±2 (secure: triangles), and Class ±1 (marginal: open 
squares/circles). It is reassuring to see that most RGB stars lie in the top right quadrant 
of the Lj vs. Lk panels, while most dwarf stars lie in the bottom left quadrant. While no 
single diagnostic is a perfect RGB/dwarf discriminant, their combination is very effective at 
separating these two populations. 
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Fig. 11. — Color-magnitude diagram of: (top) RGB stars from all fields with classes +3, 
+2, and +1 represented by filled squares, open triangles, and open squares, respectively (in 
order of decreasing certainty of classification; see §3.5), (middle) all dwarf stars with classes 
—3, —2, and —1 represented by filled circles, inverted open triangles, and open circles, 
respectively, and (bottom) all stars in our outermost field mil with the same symbols as 
above representing the different classes. The thin curves in each panel show theoretical RGB 
tracks from Vandenberg et al. (2006) for an age of 12 Gyr, [a/Fe] = 0, and metallicities 
(from left to right) of [Fe/H] = -2.3, -1.7, -1.3, -1.0, -0.7, -0.4, -0.1, +0.1, and +0.4. 
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Fig. 12. — (a) Radial velocity histograms for M31 RGB and MW dwarf stars in our two 
outermost fields m8 and mil. Confirmed RGB stars are represented by the bold solid 
histogram and secure Galactic dwarf stars by the bold dot-dashed histogram. The RGB 
PDF is shown as a solid curve, and the dwarf PDF is shown as a dashed curve (Fig. 2). 
The stars were classified as secure RGB or dwarf stars by use of the five-diagnostic method. 
The RGB and dwarf PDFs are normalized to the same area but the normalization has not 
been adjusted to match the actual number of stars. (b) Same as (a) for the DD051 
parameter (see Fig. 3). (c) Same as (a) for (V — I) vs. Nai EW for secure RGB 
stars (bold open circles) and dwarfs (crosses) in our two outermost fields. The RGB and 
dwarf two-dimensional PDF contours are shown as bold and thin lines, respectively (see 
Fig. 4). (d) Same as (c) for [Fe/H] p hot vs. [Fe/H] spec (see Fig. 6). The dot-dashed diagonal 
line shows the [Fe/H] spec = [Fe/H] p h t line. Only three RGB stars are shown; the fourth is a 
significant outlier with respect to both PDFs so this diagnostic recieved very little weight in 
the overall likelihood computation for this star (Eq. 8). 
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Fig. 13. — {Top) The (V — I)o color distribution of all stars in fields in the radial range 
R = 30-85 kpc (bold histogram). This distribution is best fit by a 75%/25% combination of 
the color distributions of secure+marginal M31 RGB (Class > 0) and MW dwarf (Class < 0) 
stars (thin dashed and thin solid histograms, respectively). (Middle) Same as (top) 
for all stars in fields in the radial range R = 85-125 kpc. The best fit in this case is an 
approximately equal mix of M31 RGB and MW dwarf stars, 45%/55%. (Bottom) Same 
as (top) for all stars in our remotest M31 halo field mil at R = 163 kpc. The mil stellar 
(V — I) color distribution generally follows that of secure+marginal MW dwarf stars. The 
(V — J)o colors of all three Class +3 stars and three of the four Class +1 stars in mil (cross- 
hatched and diagonally shaded histograms, respectively) coincide with the peak of the color 
distribution of secure+marginal M31 RGB stars. These stars form a slight apparent peak at 
(V — I)o ~ 1.5, but the peak is not statistically significant; in other words, these stars would 
not have been identified as M31 RGB stars on the basis of the color distribution alone, but 
they are identified by our likelihood method. A 5%/95% combination of M31 RGB and MW 
dwarf stars is shown. 
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Fig. 14. — (a-b) Same as Figure 12(a-b) for Class +2 M31 RGB stars as defined in §3.5 
(bold histogram). The distribution of radial velocities and DD051 parameters for these stars 
is a far better match to the RGB PDF (thin solid curve) than to the dwarf PDF (thin dashed 
curve). (c-d) Same as (a-b) for Class —2 MW dwarf stars (bold histogram). 
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Fig. 15. — (Top) Ratio of the number of marginal M31 RGB (Class +1) stars to the number 
of secure RGB+dwarf stars plotted as a function of the ratio of secure RGB to dwarf stars. 
Each data point represents one of our fields (since the ratio of secure RGB to dwarf stars 
increases with decreasing radius, the fields are roughly ordered inversely by radius along the 
x axis). The thin solid curve is a model in which the Class +1 stars are a specific mix of 
RGB and dwarf stars (a = 0.07 and b = 0.03; see §4.1.3). The Poisson error bars shown 
are based on this model. An alternate model, a = b = 0.05, is shown as a horizontal thin 
dashed line. (Bottom) Same as (top) for marginal MW dwarf (Class —1) stars. The thin 
solid curve is a model with mix parameters c = 0.03 and d = 0.15. 
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Fig. 16. — Same as Figure 12 for marginal M31 RGB (Class +1) stars (represented by bold 
histograms and open circles). The distribution of this sample in the four diagnostic plots 
indicates it consists mostly of M31 RGB stars with moderately blue colors, (V — I) ~ 1-2. 
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Fig. 17. — Same as Figure 16 for marginal MW dwarf (Class —1) stars (represented by 
bold histograms and open circles). These objects appear to be mostly MW dwarf stars with 
moderately blue colors, (V — I) ~ 1-2, judging from their distribution in the four diagnostic 
plots. 



-56- 



£0.8 



CO 
CO 

a 



0.6 



0.4 - 



O 

Q 0.2 
Q 



- 



4 - 



I 



o 



3 - 



o P 



■300 



H h 



Class 
• -3 

v -2 
■o -1 



-200 -100 

-i | i i i i | r- 





n 1 1 1 r" 



• 007 o 

o 



O • Ck • o • 
o • 



.0 • . • O 

o» I «CP* 
o • .P. 

° . . ° , °.o 
. V * 



• - • 



H h 



H H 



H h 



H h 



o 



»7 



-300 -200 -100 

Radial velocity: v hel (km/s) 



Fig. 18. — The DD051 parameter (top) and (V — I) color (bottom) plotted versus radial 
velocity for secure MW dwarf stars. As expected, dwarf stars tend to have small values of 
the DD051 parameter but there is a tail to the distribution all the way up to values near 
unity. No significant /strong trend is seen in velocity vs. DD051. There is a slight trend 
in the velocity vs. (V — I)q plot in the sense that the handful of dwarf stars with the most 
negative velocities are all very blue; the rest of the dwarf stars (v > —150 km s -1 ) span a 
large range of (V — I) colors. 



-57- 



Radial velocity: v hel (km/s) DD051 parameter 



-600 -400 -200 0.25 0.5 0.75 1 




Color: (V-I) Metallicity: [Fe/H] 



Fig. 19. — Same as Figure 12 for stars in the foreground Galactic dwarf training set with 
(V — J) < 2 (represented by the bold histograms and crosses). This color cut, shown as 
a vertical dotted line in panel (c), is designed to isolate the tail of the training set dwarf 
distribution in the portion of the (V — I)q vs. Nai EW plane where the RGB and dwarf 
PDFs overlap. Even though the stars shown in this figure are drawn from the tail of the 
(V — I)o distribution, they do not show a strong bias relative to the overall dwarf distribution 
in the radial velocity and DD051 diagnostic plots. There is obviously a correlation between 
(V — I)o and [Fe/H] phot (latter being based on CMD position); as a result the dwarfs in 
panel (d) display a bias relative to the dwarf PDF in the [Fe/H] pho t vs. [Fe/H] spcc plane. 



